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ABSTRACT 

Interest  in  structural  health  monitoring/management  (SHM)  is  attracting  lots  of  attention 
across  a  spectrum  that  ranges  from  sensor  developers  to  end  users.  The  United  States 
military,  in  particular  is  making  a  concerted  effort  to  implement  condition-based 
maintenance  (CBM)  as  a  means  of  reducing  the  life  cycle  costs  and  improving 
availability  of  various  weapons  platforms.  In  spite  of  this  effort,  the  majority  of  installed 
health  monitoring  systems  are  limited  to  rotating  machinery  such  as  engines, 
transmissions,  and  other  gear  boxes.  The  goal  of  this  workshop  was  to  bring  together 
representatives  from  military,  industry,  and  academia  covering  the  spectrum  from 
hardware  developers  to  end  users  and  platform  managers  and  have  them  discuss  issues 
that  must  be  addressed  as  SHM  systems  mature  to  the  point  that  managers  will  implement 
them.  This  paper  describes  those  discussions  and  highlights  important  issues  that  need  to 
be  addressed  as  SHM  systems  make  the  transition  from  laboratory  scale  demonstrations 
to  real-world  use. 
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INTRODUCTION 


Currently,  the  Department  of  Defense  (DoD)  in  the  United  States  is  focused  on  reducing 
the  maintenance  costs  and  increasing  the  availability  of  their  weapons  systems.  The 
result  is  a  major  push  to  implement  Condition  Based  Maintenance  (CBM)  and  significant 
success  has  been  demonstrated  in  deploying  sensors  to  monitor  engine  performance.  The 
Health  and  Usage  Monitoring  System  (HUMS)  now  deployed  on  many  rotorcraft  is  a 
prime  example  (Moorman,  2010).  Such  success  in  engine  and  transmission  monitoring 
has  not  been  matched  when  it  comes  to  structural  monitoring.  Because  of  this  slowness, 
a  group  of  structural  health  practitioners  felt  it  important  to  conduct  a  workshop  that 
would  include  the  complete  spectrum  of  people  involved  in  implementing  CBM  on 
structures  including  hardware,  software,  and  systems  developers  as  well  as  program 
managers  at  both  the  research  and  development  (R&D)  and  weapons  platform  levels.  A 
major  goal  of  this  workshop  was  to  facilitate  the  two-way  discussions  between  systems 
developers  and  end  users  to  ensure  that  developers  understand  the  requirements  and 
constraints  of  the  end  users  and  that  the  end  users  understand  the  technology  capabilities. 
Such  discussions  benefit  all  parties  by  informing  end  users  of  w  hat  is  possible  now  and  in 
the  future  and  letting  developers  know  about  high-level  constraints  that  must  be  met  for  a 
successful  technology  transition. 

The  universe  of  structural  monitoring  encompasses  a  wide  variety  of  sensor  systems, 
analysis  methods,  and  implementation  visions.  This  is  illustrated  by  the  many 
engineering  societies  that  either  organize  dedicated  conferences  or  include  sessions  on 
structural  health.  Relevant  conferences/sessions  appear  under  a  variety  of  names  such  as 
nondestructive  evaluation  (NDE),  smart  structures,  structural  health  monitoring  (or 
management),  prognostics  health  management,  or  intelligent  materials  to  name  a  few'.  In 
spite  of  the  high  levels  of  interest  within  both  the  R&D  and  program  management 
communities,  progress  in  getting  systems  tested  beyond  laboratory  settings  is  slow.  The 
consensus  among  the  organizing  committee  for  this  workshop  was  that  there  are  two 
major  factors  influencing  this  lack  of  progress:  1)  field  tests  are  expensive,  which  makes 
obtaining  funding  from  R&D  programs  unlikely,  and  2)  systems  have  not  demonstrated 
sufficient  reliability  in  the  field  to  convince  program  management  at  the  weapons 
platform  level  that  the  likelihood  of  success  is  sufficient  to  justify  the  investment. 
Because  this  is  a  circular  problem  it  will  take  movement  from  both  ends  to  create  success 
and  one  of  the  ongoing  discussions  w  ithin  the  community  and  at  this  workshop  is  about 
how  to  generate  the  needed  movement. 

The  workshop  consisted  of  five  sessions  that  targeted  problems  of  interest  to  all  parties. 
These  five  were: 

1)  Sensor  Systems:  Current  capabilities  and  needs 

2)  Data  Analysis:  Information  handling 

3)  Implementation  Issues 

4)  Performance  Validation  and  Certification 

5)  Cost  Benefit  Analysis 
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Sensor  systems  are  the  fundamental  building  block  in  assessing  structural  health.  Thus, 
the  opening  session  was  devoted  to  describing  w  hat  sensors  are  capable  of  both  now  and 
in  the  future.  However,  sensors  are  only  valuable  when  the  data  collected  from  the 
sensors  is  converted  into  the  information  needed  by  the  user.  There  are  significant 
challenges  in  this  arena  that  span  the  range  from  the  technological,  e.g.  how  much  data  is 
generated  and  how  do  you  analyze  and  store  it,  to  human  factors,  where  ‘who  knows 
what?'  and  ‘when  should  they  be  told?’  are  key  programmatic  decisions.  The  session  on 
implementation  issues  dealt  with  how  to  bridge  the  gap  between  lab  level  results  and  the 
field  reliability  desired  at  the  program  level.  It  is  worth  reiterating  that  there  is  general 
agreement  that  this  is  the  area  where  most  transition  failures  occur.  Performance 
validation  and  certification  is  a  step  that  any  new  technology  will  have  to  achieve  before 
it  becomes  generally  accepted.  However  structural  health  systems  face  a  daunting 
challenge  because  the  structures  (e.g.  ships,  ground  vehicles,  and  aircraft)  are  too 
expensive  to  test  a  statistically  significant  number.  Thus  other  ways  of  making 
statistically  valid  tests  that  can  be  compared  across  sensors  and  analysis  methods  are 
needed.  Finally,  should  a  structural  health  system  make  it  past  the  hurdles  described  in 
the  first  four  sessions,  the  final  say  in  implementation  will  probably  be  determined  by 
some  kind  of  cost  benefit  analysis. 

In  the  sections  that  follow,  we  will  describe  each  topic  and  present  the  discussion, 
analysis,  and  any  conclusions.  We  expect  the  conclusions  drawn  from  these  discussions 
to  be  the  starting  point  for  continued  discussion  within  the  structural  monitoring 
community. 
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CURRENT  CAPABILITIES  AND  NEEDS 


Background 

The  main  goal  of  this  session  was  to  provide  both  sensor  systems  developers  and 
potential  end  users  with  some  general  guidance  as  to  how  advanced  different  sensor 
systems  are  with  respect  to  their  uses  for  SUM.  As  anyone  who  has  attended  conferences 
w  ith  sessions  devoted  to  SHM  knows,  there  are  both  a  wide  variety  of  sensor  types  and 
numerous  specific  implementations  of  each  sensor  type  touted  as  having  major 
capabilities  for  structural  monitoring.  Because  of  this  diversity,  a  thorough  discussion  of 
sensor  capabilities  could  easily  occupy  many  days  and  result  in  a  book  length  description. 
Thus,  this  session  was  limited  to  a  general  description  of  capabilities  broken  down  by 
sensor  type  and  upper  frequency  limit.  Beneath  the  admittedly  arbitrary  limits  used 
during  this  session  and  shown  in  Table  2,  there  is  a  lot  of  underlying  information  that 
should  be  investigated  by  anyone  interested  in  implementing  a  SHM  system  or  in 
developing  new  or  improved  sensing  capabilities.  One  place  to  start  such  an  investigation 
is  by  reading  section  2.  Sensing  System  Design  Considerations  for  SHM.  in  the  report 
from  Los  Alamos  National  Laboratory  describing  their  workshop  on  “Energy  Harvesting 
for  Structural  Health  Monitoring  Sensor  Networks"  (Park  et.  al.,  2007).  This  portion  of 
the  report  provides  a  general  description  of  sensor  types  and  capabilities  focused  on 
SHM.  and  some  details  on  a  few  specific  sensor  types  such  as  accelerometers,  fiber 
Bragg  gratings  (FBGs)  and  PZT  sensors. 

We’ve  chosen  to  use  a  draft  revision  of  the  DoD  Technology  Readiness  Levels  (TRLs)  as 
listed  in  Table  1  to  characterize  the  development  stages  of  a  sensor  system  (Graettinger 
et.  al..  2002).  Note  that  this  table  differs  from  the  standard  DoD  or  NASA  TRL  list  in 
that  it  separates  hardware/subsystems  (HW/S)  from  software  (SW).  One  major  driver  for 
use  using  the  draft  version  TRLs  is  because  most  weapons  platforms  already  have  an 
overall  software  environment  into  which  SHM  software  will  need  to  be  integrated.  Such 
integration  represents  a  huge  effort  in  software  development  and  testing  that  would 
include  specified  communications  protocols  for  the  SHM  systems  as  well  as 
modifications  to  the  platform  level  software  for  SHM  displays  and  reports.  The  effort 
(and  expense)  required  suggests  that  early  SHM  implementations  may  be  best  as  stand¬ 
alone  systems.  The  TRL  level  table  alludes  to  the  software  integration  effort  in  the 
distinction  between  TRL  levels  5  and  6  for  software  by  noting  that  algorithms  run  on  a 
processor  with  “characteristics  expected  in  the  operational  environment"  (TRL  5)  as 
opposed  to  algorithms  run  on  a  “processor  of  the  operational  environment”  (TRL  6).  It 
should  also  be  noted  that  the  tabulation  of  sensor  systems  in  Table  2  was  focused  on 
hardware  status,  not  software  status  and  does  not  take  into  account  the  preceding 
discussion. 
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Sensor  Capabilities 

Table  2  provides  a  description  of  the  development  levels  for  a  number  of  systems  that 
sense  mechanical  parameters  using  distinctions  based  on  sampling  rate.  The  systems 
designations  were  chosen  based  on  distinctions  that  are  often  seen  at  conferences. 
Conventional  systems  cover  the  sensors  and  sensing  approaches  that  have  been  used 
routinely  for  more  than  a  decade.  This  includes  resistive  strain  gauges,  accelerometers, 
linear  variable  displacement  transformers  (LVDTs),  etc.  These  systems  can  also  be 
thought  of  as  requiring  multiple  w  ire  electrical  connections  and  signal  conditioning  for 
each  sensor.  While  such  systems  have  a  long  history  of  quality  measurements,  the 
extensive  wiring  harness,  bulky  signal  conditioning,  and  sensitivity  to  electromagnetic 
interference  make  them  unlikely  candidates  for  on  board  SHM.  Fiber  optic  sensors 
encompass  a  w  ide  array  of  types.  Here  we  have  focused  on  multiplexed  systems  w  here 
many  sensors  can  be  readily  incorporated  in  a  single  optical  fiber.  These  include  the 
optical  scattering  approaches  that  rely  on  Rayleigh.  Raman,  or  Brilluouin  scattering 
where  the  fiber  itself  is  the  sensor  and  the  limits  on  multiplexing  lie  in  the  time  duration 
of  the  light  pulse  or  the  capabilities  of  the  digitizer  (Kersey,  1991).  The  other  main  type 
of  fiber  optic  sensor  is  the  fiber  Bragg  grating  (FBGs)  where  sensors  are  manufactured 
into  the  fiber  at  specific  locations,  with  predetermined  capabilities  such  as  length  and 
wavelength  (Kersey  et.  al.,  1997).  There  are  other  types  of  fiber  optic  sensors  that  can  be 
multiplexed  such  as  extrinsic  Fabry-Perot  interferometers  (EFP1),  but  they  are  rarely  used 
for  structural  monitoring.  We  have  not  included  the  interferometric  fiber  optic  sensors, 
which  can  offer  much  higher  sensitivities  and  data  rates,  but  currently  require  individual 
demodulation  for  each  sensor  resulting  in  readout  hardware  that  rivals  conventional 
systems  in  size  and  weight.  Piezoelectric  (PZT)  strain  gauges  and  accelerometers  have 
been  around  for  decades  and  thus  could  fall  into  the  conventional  systems  category,  but  it 
seems  more  appropriate  to  include  them  with  the  PZT  based  ultrasonics  and  impedance 
methods  that  are  currently  being  investigated  for  detecting  early  stage  damage  such  as 
sub-millimeter  cracks  and  corrosion  since  they  are  rarely  used  conventionally.  We 
include  MEMS  as  a  separate  category  because  while  the  sensing  methods  are 
conventional,  the  size  reductions  and  possibility  of  including  local  digitization  can 
overcome  several  of  the  major  barriers  to  real  world  implementations.  As  a  final 
category,  we  include  energy  harvesting  systems.  Again,  the  sensors  themselves  are  based 
on  well  known  sensing  methods:  the  challenge  here  is  to  create  a  system  that  eliminates 
the  electrical  wires  and  includes  the  signal  conditioning  and  digitization  in  a  tiny  node 
using  power  generated  at  or  near  the  sensor  node.  Currently,  limits  in  power  generation 
rates  and  the  power  use  for  radio  communications  have  limited  data 
acquisition/transmission  rates  to  well  below  1  Hz. 

The  horizontal  categorization  axis  was  chosen  to  be  data  acquisition  rate.  This  choice  was 
based  on  "conventional  wisdom”.  First,  it  is  widely  believed  that  the  length  sensitivity  of 
a  method  scales  inversely  with  frequency.  That  is,  low  frequency  methods  can  only  detect 
large  amounts  of  damage  while  high  frequency  methods  can  detect  much  smaller  damage 
or  even  possible  damage  precursors.  One  way  of  visualizing  this  concept  is  that  the 
length  scale  of  the  damage  needs  to  cover  a  significant  portion  of  the  wavelength  used  for 
the  detection,  when  the  size  of  the  damage  is  a  small  fraction  (say  <  0.1%)  of  the 
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wavelength,  even  interacting  with  the  maximum  amplitude  region  of  the  detection  wave 
results  in  minimal  signal.  The  result  is  no  change  in  the  measurement  and  no  detection  of 
damage.  Based  on  this  concept,  we  expect  that  low  frequency  methods,  and  we'll 
arbitrarily  pick  500  Hz  for  a  data  rate,  should  not  be  considered  for  many  type  of  damage 
detection  but  should  work  well  for  loads  (fatigue)  monitoring,  shape  sensing,  and  for 
modal  analysis.  Although  we  will  not  discuss  them  here,  there  is  an  extensive  literature 
investigating  approaches  such  as  tuned  excitation,  nonlinearity  detection,  etc.  that  may 
provide  improved  damage  detection  capabilities  while  using  low  frequency  systems. 
Second,  as  indicated  above,  the  structural  information  provided  by  a  measurement  system 
depends  strongly  on  the  data  acquisition  rate.  The  low  frequency  data  critical  for  fatigue 
monitoring  or  global  shape  sensing  in  many  instances  cannot  be  provided  by  high 
frequency  systems  because  they  don’t  offer  the  high  accuracy  and  long  term  stability 
needed  for  static  or  quasi-static  measurements.  Similarly,  the  kinds  of  self-calibration 
required  for  static  measurements  preclude  the  high  frequency  operations  that  may  provide 
early  damage  detection  capabilities.  In  between  are  measurements  of  impulsive  events 
such  as  wave  slamming  in  ships  and  bird  or  stone  impacts  in  aircraft  that  require  a 
moderate  frequency  response  (5  -  20  kHz)  when  it  is  important  to  capture  the  peak  forces 
accurately.  Finally,  the  upper  frequency  distinction  (<500  kHz  vs  >1  MHz)  comes  from 
the  sense  that  Lamb  wave  detection  focused  on  the  lowest  modes  (Ao,  and  So)  and 
acoustic  emission  work  will  occur  at  frequencies  below  about  500  kHz  and  that  there  will 
be  significant  hardware  and  performance  differences  as  frequency  capabilities  are 
extended  above  a  Megahertz. 

Needs 

Because  of  the  DoD-wide  push  for  condition  based  maintenance  (CBM  and  CBM+), 
there  is  strong  interest  in  structural  monitoring  and  automated  damage  detection  on  the 
part  of  end  users.  In  fact,  several  weapons  platforms  that  are  currently  being  developed 
(such  as  Joint  Strike  Fighter  (JSF)  and  Littoral  Combat  Ship  (LCS)  have  structural 
monitoring  requirements.  The  specifics  of  how  structural  monitoring  will  be  implemented 
is  not  clear,  although  in  the  case  of  JSF  it  w  ill  include  few  if  any  structural  sensors  and 
the  fatigue  monitoring  will  rely  on  recorded  flight  parameters.  The  lack  of  clarity  in 
implementation  relates  strongly  to  sensor  system  development  and  the  lack  of  confidence 
on  the  part  of  program  managers  and  platform  developers  that  appropriate  sensor  systems 
have  reached  an  adequate  state  of  development  (e.g.  hardware  >  TRL  6).  Thus,  the  needs 
discussion  was  focused  on  how  to  provide  convincing  evidence  of  hardware  readiness 
and  reliability.  The  consensus  was  that  there  are  two  critical  aspects  in  demonstrating 
that  a  sensor  system  has  reached  the  maturity  needed  for  inclusion  in  a  weapons  platform. 
These  are: 

1.  Reliable  performance  mast  he  demonstrated  through  incremental  demonstrations 

Once  a  system  has  shown  strong  promise  in  traditional  laboratory  tests  such  as  crack 
growth  from  an  EDM  notch  on  a  dogbone  test  article,  impact  damage  on  flat  plates,  lamb 
wave  detection  of  holes  drilled  in  plates,  etc.,  it  should  be  tested  on  structures  that  are 
larger  and  more  complex  using  more  realistic  types  of  damage.  Examples  might  be  large 
panels  with  lap  joints  and/or  stiffeners,  curved  surfaces,  or  materials  with  varying 
thickness.  Analysis  methods  and  sensors  should  also  be  tested  for  temperature  sensitivity. 
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Success  at  this  stage  indicates  TRL  4-5  depending  on  the  degree  of  structural  complexity 
and  the  sophistication  of  the  hardware.  It  is  worth  noting  that  multiple  DoD  agencies 
have  ongoing  SBIR  efforts  with  similar  goals. 

The  next  performance  stage  involves  limited  field-testing.  On  the  hardware  side,  this  can 
require  dramatic  changes  in  hardware  as  systems  that  work  well  under  the  ±2  °C, 
constant  humidity  conditions  in  a  laboratory  get  challenged  by  cooking  for  hours  in  direct 
sun  at  35  C,  experiencing  condensing  humidity  or  rain,  freezing,  dust,  etc.  In  fact,  a 
routine  problem  in  moving  from  the  lab  into  the  field  is  the  difficulty  of  reading  a  laptop 
screen  in  direct  sun.  One  benefit  of  limited  field-testing  is  that  it  keeps  costs  down  by 
minimizing  the  amount  of  time  that  personnel  have  to  be  deployed  and  the  associated 
travel  costs.  Following  a  very  successful  limited  field  test  hardware  can  be  considered  to 
have  reached  early  TRL  6  and  program  managers  may  be  willing  to  consider  its  use  in 
their  efforts. 

The  final  demonstration  of  hardware  readiness  is  a  long-term  field  test  where  the  system 
is  operated  by  military  personnel  with  occasional  advice/oversight  from  the  developer. 
The  fact  that  the  system  must  be  operated  by  military  personnel  makes  including 
automated  operating  procedures  and  straightforward  user  interfaces  necessary.  A 
successful  long-term  field  test  provides  strong  support  that  a  system  is  ready  for  real- 
world  use  and  will  facilitate  the  inclusion  of  said  system  in  the  platform  development 
process. 

It  is  strongly  recommended  that  a  sensor  system  developer  have  successful  results  from  at 
least  one  of  these  types  of  demonstrations  before  they  approach  a  program  office  offering 
a  solution  for  that  particular  platform. 

2.  Performance  results  must  be  comparable  across  sensor  systems 

Because  each  sensor  system  offers  specific  advantages  and  drawbacks,  it  is  important  that 
the  end  users  be  able  to  evaluate  performance  with  respect  to  their  specific  requirements. 
Thus,  program  offices  must  be  able  to  evaluate  system  performance  demonstrations 
across  multiple  sensor  systems  with  respect  to  problems  that  are  specific  to  that  platform. 
In  the  nondestructive  evaluation  (NDE)  world,  one  way  of  addressing  this  issue  is 
through  probability  of  detection  (POD)  measurements.  However,  proper  POD  results  in 
NDE  require  measurements  on  large  numbers  of  test  samples  based  on  a  factorial  or 
partial  factorial  experimental  design  (M1L-HDBK-1823,  1999).  Such  designs  would  be 
cost  prohibitive  in  structural  monitoring.  Thus,  the  SHM  world  needs  to  come  up  with  a 
comparable  approach  that  is  based  on  statistical  testing.  Another  approach  that  describes 
both  POD  and  probability  of  false  alarms  (PFA)  in  a  single  plot  is  the  receiver  operating 
characteristic  (ROC)  curve  (Swets,  1988).  However,  again  a  best  practices  evaluation 
would  be  cost  prohibitive.  Currently,  research  is  underway  looking  into  combining 
measurements  with  modeling  (model  assisted  POD  or  MAPOD)  in  an  effort  to  control 
the  costs  of  POD  studies  while  minimizing  any  reduction  in  statistical  quality 
(Thompson.  2001).  Solving  the  problem  of  getting  a  statistically  valid  assessment  is  a 
critical  step  in  the  acceptance  of  SI  1M  systems  by  the  DoD. 

Another  way  of  providing  sensor  system  comparison  is  to  hold  joint  demonstrations. 
Because  of  the  difficulties  involved  in  setting  up  known  and  reproducible  damage  in  a 
complex  structure  or  in  conducting  field  tests,  it  can  be  time  and  cost  effective  to  include 
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multiple  systems  in  a  single  round  of  tests.  Since  all  systems  are  looking  at  the  same 
events,  performance  comparisons  are  simplified. 

Demonstrating  System  Performance 

The  emphasis  on  robust  demonstrations  of  system  performance  in  the  last  section  makes 
it  important  that  sensor  system  developers  become  aware  of  any  upcoming  large-scale 
structure  demonstrations  so  that  they  can  explore  the  possibilities  of  piggybacking  sensor 
performance  on  these  tests.  Learning  about  such  demonstrations  requires  knowing  the 
right  people  and  asking  the  right  questions,  which  is  not  easy  for  someone  outside  of  the 
military  R&D  structure.  It  is  also  possible,  usually  by  collaborating  with  DoD 
researchers,  to  persuade  program  managers  to  set  up  shorter  capability  demonstrations 
when  they  target  known  problems  or  desired  capabilities. 

In  the  Army  and  Navy  (including  the  Marines)  program  offices  exist  for  specific  weapon 
platforms.  In  the  Navy,  this  means  finding  the  PMA  (A  for  aircraft)  or  PMS  (S  for  ships) 
responsible  for  a  particular  platform,  e.g.  the  navy’s  PMA-253  is  responsible  for 
developing  a  new  heavy  lift  helicopter.  Navy  supporting  labs  include  the  Naval  Research 
Laboartory.  the  Surface  Warfare  Centers,  Undersea  Warfare  Centers,  Naval  Air  Stations, 
etc.  In  general,  it  is  the  scientist/engineers  at  these  centers  who  know  what  the  program 
offices  are  interested  in  and  the  history'  of  investigated  solutions.  The  Naval  Research 
Lab  (NRL)  is  the  navy’s  basic  research  lab  and  while  its  researchers  individually  have 
knowledge  and  connections  with  the  program  offices,  those  connections  are  generally  not 
as  strong  as  the  Warfare  Centers  and  Air  Stations.  Similarly,  the  Army  has  Research  and 
Development  Engineering  Centers  (RDECs  or  RDCs)  with  good  connections  to  the 
program  offices.  In  the  Army,  the  Army  Research  Laboratory  (ARL)  does  both  basic  and 
applied  research  with  the  quality  of  a  researcher's  connections  to  program  offices 
depending  on  the  individual.  A  system  developer's  chances  of  success  in  getting 
program  office  funding  arc  significantly  enhanced  through  working  with  military 
scientist/engineers  towards  solutions  that  will  solves  a  program  office  problem.  Thus, 
getting  support  for  a  large-scale  laboratory  or  small  field  demonstrations  in  the  Army  or 
Navy  usually  requires  collaborating  with  DoD  researchers  and  approaching  specific 
program  offices. 

In  the  Air  Force,  it  is  the  Systems  Program  Offices  (SPOs)  overseeing  particular 
platforms  that  set  up  large-scale  tests.  Most  of  the  Air  Force's  research  occurs  at  the  Air 
Force  Research  Laboratory  (AFRL)  with  most  researchers  at  Wright-Patterson  Air  Force 
Base,  near  Dayton.  OH  and  a  contingent  in  the  Space  Vehicles  Directorate  at  Kirkland 
Air  Force  Base,  near  Albuquerque,  NM.  It  is  the  scientists/engineers  at  these  locations 
who  will  be  aware  of  upcoming  tests  on  a  case-by-case  basis.  Mark  Derriso  at  AFRL, 
Dayton  is  setting  up  a  large-scale  structural  health  monitoring  test  bed  with  the  goal  of 
providing  a  facility  where  SHM  capabilities  can  be  demonstrated  and  compared.  Much 
of  the  design  and  construction  of  that  facility  have  been  completed,  but  the  details  of  how 
access  will  be  granted  and  how  funding  for  such  tests  will  work  are  still  being  discussed. 
Once  those  are  finalized,  the  information  will  be  broadcast  to  the  SHM  community. 

Capabilities  and  Needs  Summary 
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In  summary,  while  it  is  clear  that  the  military’s  push  toward  condition  based  maintenance 
has  unlocked  the  door  for  structural  sensing,  it  is  not  clear  that  sensing  systems  with 
suitable  weight  and  power  characteristics  have  demonstrated  appropriate  levels  of 
readiness.  The  consensus  was  that  there  are  multiple  sensor  systems/types  that  are 
approaching  the  readiness  levels  that  can  facilitate  deployment  on  military  platforms  for 
structural  monitoring  but  that  most  of  these  systems  need  to  undergo  additional  testing  to 
demonstrate  capabilities  under  real-world  conditions.  It  is  also  clear  that  both  sensor 
reliability  and  sensitivity  will  need  to  be  demonstrated  before  a  system  can  be  accepted. 

Sensor  systems  can  be  categorized  into  various  types,  which  we  subdivided  by  data 
acquisition  rates.  The  data  rates  are  important  in  structural  monitoring  because  low  data 
rates  limit  systems  to  loads  monitoring  and/or  modal  analysis,  neither  of  which  provide 
high  sensitivity  to  small  amounts  of  local  damage.  Intermediate  data  rates  will  be  needed 
for  impact  detection,  wave  slamming,  and  other  impulsive  events.  High  data  rates  offer 
the  ability  to  detect  ultrasonic  waves,  which  can  be  useful  for  detecting  the  early  stages  of 
damage. 


INFORMATION  HANDLING 


Session  Goal 

The  goal  of  the  Information  Handling  Session  was  to  identify  and  discuss  critical 
integrated  structural  health  monitoring  (ISHM)  challenges  in  processing  data  obtained 
from  sensor  measurements.  Such  challenges  include  storing  and  retrieving  raw  data, 
extracting  relevant  damage  features  (converting  data  into  information),  processing 
feature-based  information,  detecting  the  presence  of  damage,  and  estimating  pertinent 
damage  parameters.  The  Information  Handling  participants  concentrated  their  discussion 
on  three  main  issues:  data  management,  data  standardization,  and  data  processing 
architectures. 


Data  Management 

The  DoD  has  multiple  health  monitoring  systems  currently  deployed,  focused  largely  on 
the  health  of  engines  and  drive  trains  as  well  as  their  use.  These  systems  collect  vast 
amounts  of  data  from  installed  sensors  or  during  scheduled  inspections,  making  data 
management  one  of  the  largest  problems  in  already  Fielded  systems.  Data  management 
issues  include  storage  and  transmission  costs,  maintaining  data  integrity,  and  adequate 
processing  to  convert  data  to  useful  information.  In  a  keynote  presentation,  the  Army 
indicated  that  it  has  installed  condition-based  maintenance  (CBM)  functionality  on  1458 
of  its  3366  aircraft  (Smith,  2010).  Specifically,  these  aircraft  were  wired  for  rotor 
smoothing,  drive  train  health,  exceedance  monitoring,  structural  health,  engine  health,  or 
logbook  interfacing.  However,  the  CBM  systems  are  generating  terabytes  to  petabytes  of 
data  every  month.  Storing,  translating  and  processing  such  large  amounts  of  data  is 
extremely  expensive,  since  current  systems  have  limited  onboard  processing  capability 
and  the  data  is  generally  analyzed  offline.  In  addition,  since  the  data  changes  hands  up  to 
twenty-three  times  between  the  collection  and  analysis  stages,  ensuring  data  integrity  is 
almost  impossible.  Finally,  the  data  collected  does  not  always  contain  useful 
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information.  In  order  to  reduce  the  cost  of  data  transmission  and  storage,  lossy 
compression  was  used  as  a  data  management  technique.  However,  much  broader  data 
management  methodologies  need  to  be  used  to  achieve  successful  ISHM  in  real  systems 
in  the  near  future. 

The  Session  participants  had  several  suggestions  on  how  to  improve  data  management  in 
health  monitoring.  These  suggestions  include; 

■  Although  high  performance  computing  has  been  used  to  solve  computational 
problems  such  as  large-scale  matrix  inversion  using  supercomputers  and  computer 
clusters,  it  is  now  being  modified  to  deal  w  ith  the  new  technological  demands  of  data 
intensive  processing.  As  a  result,  within  the  next  five  years,  high  performance 
computing  technology  may  be  available  to  alleviate  the  data  management  problems 
we  are  facing  today. 

■  There  are  other  research  areas  that  are  more  mature  in  dealing  with  very  large  data 
sets  than  the  ISHM  area.  It  would  be  beneficial  to  study  what  methodologies  these 
areas  employ  in  solving  their  data  management  problems.  Examples  of  these  research 
areas  include  monitoring  and  investigating  credit  card  fraud,  data  collection  and 
analysis  by  the  US  census  bureau,  and  outbreak  monitoring  by  the  US  Centers  for 
Disease  Control  (CDC). 

■  One  approach  to  data  management  is  replacing  conventional  processing  with 
intelligent  processing  methodologies.  Examples  of  these  methodologies  include:  (a) 
using  compressive  sampling,  instead  of  conventional  sampling,  to  reduce  the  amount 
of  data  collected  by  the  sensors;  (b)  devising  techniques  to  intelligently  discard  the 
unimportant  data  before  transmission:  (c)  developing  data  fusion  techniques  when 
multiple  sensors  are  used:  (d)  employing  adaptive  processing  to  leam/gain  knowledge 
from  past  data  in  order  to  improve  future  knowledge;  (e)  automating  the  system  to 
reduce  the  need  for  large  data  sets  and  more  importantly  removing  multiple  human 
interactions  (and  thus  multiple  sources  of  error  due  to  data  handling). 


Data  Standardization 

Currently,  there  is  no  process  for  developing  and  agreeing  upon  technical  standards  in  the 
area  of  health  monitoring.  The  lack  of  a  common  data  interface  is  a  challenge  that  current 
systems  are  facing,  and  the  diversity  in  data  collection  platforms  and  data  formats  makes 
comparison  difficult.  As  a  result,  it  would  be  good  to  have  a  steering  committee  or 
working  group  that  can  recommend  or  design  a  standard  data  format.  For  example,  this 
would  help  us  learn  not  to  collect  data  that  was  not  useful  and  collect  new  types  of  data 
that  could  provide  new  or  more  meaningful  information.  The  structural  health 
monitoring  community  has  a  working  group  called  the  Aerospace  Industrial  Steering 
Committee  (AISC)  that  is  already  looking  into  various  other  implementation  issues.  Data 
standardization  might  also  be  included  under  their  auspices.  In  addition,  session 
participants  suggested  that  we  should  investigate  data  standardization  practices  in  other, 
more  mature,  research  areas. 
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Data  Processing  Architectures 

It  is  important  that  ISHM  methodologies,  algorithms,  and  standards  developed  today 
remain  compatible  with  the  expected  advancements  in  hardware  configurations.  In 
particular,  algorithmic  solutions  need  to  be  proposed  that  will  take  into  consideration 
possible  architectures  that  will  be  available  in  the  future  to  implement  those  solutions. 
The  hardware  five  years  from  now,  built  for  dealing  w  ith  large  amounts  of  data,  can  be 
quite  different  from  what  we  current  have  available.  Also,  advanced  algorithms  and 
software  must  be  designed  to  take  advantage  of  newr  hardware  configurations.  Note  that  a 
significant  body  of  work  exists  for  non-ISHM  systems  that  could  be  adapted  for  use  in 
ISHM  systems.  There  are  many  applications  where  tools  have  already  been  developed  to 
deal  with  large  amounts  of  data  with  efficiency  and  speed.  That  experience  should  be 
drawn  upon  and  used  for  guidance.  We  are  currently  capable  of  identifying  only  5-10% 
of  failure  modes  using  CBM;  the  target  for  the  next  five  years  is  to  increase  that  number 
to  90%. 


IMPLEMENTATION  ISSUES  OF  CURRENT  RESEARCH 

The  discussions  on  implementation  issues  focused  around  three  main  topics,  each 
summarized  below.  During  the  discussion  an  additional  topic  was  mentioned,  the 
definition  of  implementation.  Numerous  ideas  for  an  appropriate  definition  were 
discussed.  These  include:  1)  Implementation  occurs  when  an  SUM  system  reaches  TRL 
8.  2)  A  system  is  implemented  only  when  it  is  used  on  multiple  aircraft,  ships,  or  ground 
vehicles.  3)  Implementation  is  when  the  system  has  been  institutionalized  e.g. 
maintenance  credit  established.  Suffice  it  to  say  that  SHM  implementation  can  only  occur 
when  it  is  technologically  mature  and  being  utilized  as  an  integral  part  of  a  larger  system. 
The  following  discussion  topics  and  ideas  address  issues  associated  with  the 
implementation  of  SHM. 

Risks  of  SHM  Implementation 

In  general  there  are  two  major  sources  of  risk  in  implementing  any  new'  system.  These 
are:  1)  the  maturity  and  reliability  of  the  technology,  and  2)  the  programmatic  risk 
associated  w  ith  the  cost  and  schedule  for  integrating  the  technology  into  the  final  system. 


In  order  to  address  risks  associated  with  maturity  and  reliability,  iterative  approaches  are 
needed  to  incrementally  demonstrate  the  technology.  While  this  is  true  and  small 
demonstrations  are  positive,  to  convince  decision  makers,  step  change  demos  are  needed 
to  truly  show  the  maturity  and  reliability  of  SHM  systems.  The  Army’s  CBM  program,  to 
collect  and  analyze  rotorcraft  vibration  data,  has  shown  the  value  of  health  monitoring  for 
a  few  applications.  High  confidence  in  reliability  and  effectiveness  is  needed  in 
addressing  all  the  important  failure  modes.  If  some  important  failure  modes  are  left  out,  it 
may  not  be  worth  the  investment.  The  use  of  mature  and  reliable  SHM  provides  an 
opportunity  to  reduce  safety  factors  and  therefore  system  weight.  The  weight  and  space 


requirements  of  an  SUM  system  are  likely  to  be  a  risk  as  many  military  platforms  are 
already  weight/space  constrained. 

In  order  to  overcome  the  programmatic  risks,  there  must  be  a  push  and/or  pull  for  the 
technology  and  it  needs  to  be  clearly  shown  that  SHM  can  reduce  the  maintenance 
burden.  Push  and/or  pull  needs  to  come  from  generals,  maintainers  etc.  One  of  the  main 
reasons  to  implement  SHM  is  to  reduce  the  maintenance  burden,  which  in  turn  can 
reduce  life  cycle  cost.  Evidence  of  the  life  cycle  cost  reduction  and  push/pull  for  the 
technology  may  be  able  to  convince  program  managers  that  the  cost  and  schedule  risk 
associated  with  SHM  are  manageable. 

Transition  from  Traditional  Maintenance  to  Health  Monitoring-logistics  Plan 

Logistics  plans  will  be  critical  to  transition  from  traditional  maintenance  procedures  to 
health  monitoring  based  maintenance  procedures.  One  important  aspect  of  the  transition 
will  be  the  need  to  work  more  closely  with  maintainers.  During  the  transition  phase  it 
may  be  necessary  to  have  the  two  maintenance  approaches  working  in  parallel.  In 
addition,  modifications  to  maintenance  contracts  and  manuals  will  be  needed.  In  order  for 
a  full  transition  to  occur,  policy  changes  will  be  required  for  the  logistics  infrastructure. 

Existing  Systems  and  Platforms  Amenable  for  Demonstration  to  Ease  Integration 

Successful  demonstrations  are  very  important  if  implementation  of  SHM  is  to  occur.  Two 
basic  approaches  to  demos  have  been  attempted.  One  approach  is  to  instrument  numerous 
platforms  and  take  data,  the  amount  of  which  can  be  very  large.  The  data  can  then  be 
analyzed  by  various  methods  to  determine  if  anomalies  are  present.  It  was  pointed  out 
that  for  this  approach  to  be  successful  management  of  expectations  is  critical,  i.e.  success 
criteria  must  be  defined  in  advance.  Previous  large-scale  demos  have  failed  due  to  ill- 
defined  success  criteria.  The  second  approach  is  to  utilize  smaller  iterative  demos  to  build 
up  to  a  successful  full-scale  demonstration  system.  Both  approaches  can  be  valuable.  For 
example,  the  C-17  has  been  targeted  for  a  potential  demonstration  platform  and  funding 
avenues  are  being  explored  by  the  Air  Force.  The  C-17  demo  is  planned  to  be  more  like 
the  first  approach,  that  is:  instrument,  collect,  analyze  date,  and  learn  as  you  go.  It  was 
also  suggested  that  corrosion  detection  and  unmanned  aerial  vehicles  (UAVs)  may  be 
good  areas  for  demonstrations.  Other  important  aspects  discussed  included  a  central 
repository  for  test  results,  cooperation  among  DoD  laboratories,  and  “marketing"  of  SHM 
technology. 

Conclusion 

For  SHM  to  be  implemented,  i.e.  institutionalized  with  logistics  and  maintenance  plans 
and  maintenance  credits,  push  from  maintainers  and/or  pull  from  high  ranking  officials 
will  be  needed  along  with  a  change  in  the  maintenance  culture.  The  push/pull  is  not  likely 
without  successful  iterative  small-scale  demos  in  addition  to  large-scale  data  intensive 
demos.  It  is  critical  for  these  demos  to  have  well  defined  success  criteria,  a  central 
repository  for  demonstration  results,  and  cooperation  between  the  DoD  labs. 
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PERFORMANCE  VALIDATION  AND  CERTIFICATION 


The  session  discussion  was  focused  on  methods  for  the  validation  and  certification  of 
candidate  systems  and  the  identification  of  a  process  for  transitioning  research  to 
practice.  It  was  recognized  that  researchers  would  have  to  understand  the  real-world 
challenges  and  needs  of  the  application  and  to  communicate  and  work  closely  with  the 
end-users.  It  is  critical  that  the  research  be  coupled  with  the  requirements. 

It  was  suggested  that  it  would  be  better  to  proceed  in  small  steps  and  first  demonstrate  the 
advantage  of  the  proposed  approach  on  a  specific  component.  For  example,  while  ageing 
aircraft  are  a  current  problem  of  high  interest,  there  are  currently  no  standard  test 
specimens  on  representative  structures.  The  Federal  Aviation  Administration  (FAA)  has 
standard  NDE  test  specimens.  While  there  are  no  standard  specimens  within  the  military, 
outer  wing  panel  tests  have  been  used  upon  request.  Mark  Derriso  of  the  Air  Force 
Research  Laboratory  (AFRL)  is  working  on  the  development  of  a  standard  test  bed. 

The  real  proof  of  validation  for  justifying  implementation  would  be  to  demonstrate  that 
the  method  meets  current  performance  requirements  on  an  actual  structure,  with  a 
building  block  process  that  considers  variability  issues  of  real  data  and  provides 
quantitative  results  (e.  g.  a  probability  of  detection  study  for  the  FAA).  It  was  pointed  out 
that  some  organizations  such  as  the  FAA  could  be  in  a  good  position  to  validate  the 
capabilities  of  a  test  system. 

A  comparison  would  also  be  required  w  ith  existing  methods.  One  example  of  an  existing 
system  is  the  Health  and  Usage  Monitoring  System  (HUMS)  developed  for  use  as  a 
diagnostic  tool  for  rotating  machinery  (Moorman,  2010).  However,  based  on  maximum 
flight  loads  instead  of  probabilistic  loads,  the  HUMS  system  is  not  FAA  certified  and 
does  not  fit  currently  with  the  Navy’s  needs.  Nevertheless,  HUMS  is  an  existing  health 
monitoring  system  that  is  being  implemented  and  can  serve  as  a  model  for  other 
candidate  systems.  Indeed,  it  should  be  pointed  out  that  the  Navy  does  have  some 
HUMS  systems  and  is  evaluating  their  performance  and  use. 

A  full  qualification  plan  and  cost-benefit  analysis  is  required  before  the  proposed 
condition  based  maintenance  method  can  be  fully  adopted.  To  be  feasible,  the  health 
monitoring  framework,  which  includes  installation,  use,  and  maintenance,  must  be 
cheaper  than  replacing  parts!  To  complete  validation,  a  handbook,  such  as  M1L-HDBK- 
1823  (1999)  is  required  that  provides  guidance  (not  standards)  to  users. 

Contacting  the  appropriate  agency  is  a  step  toward  demonstrating  advantage  and 
feasibility.  Each  service  has  a  certification/qualification  team,  for  example,  the 
Aeronautical  Systems  Center  (ASC)  for  the  Air  Force  and  the  System  Program  Office 
(SPO)  for  the  Navy.  However,  this  step  is  necessary  only  after  multiple  successful 
demonstrations  and  is  likely  to  be  overseen  by  the  program  office. 
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COST  BENEFIT  ANALYSIS 


The  scope  and  context  for  the  Cost  Benefit  Analysis  session  was  to  consider  attributes  for 
a  CBM  approach  to  health  management.  The  session  participants  explored  the  following 
three  questions  to  help  focus  the  discussion: 

■  What  is  cost  and  benefit? 

■  Are  cost  benefit  studies  beneficial? 

■  What  limits  the  use  of  cost  benefit  studies? 

What  are  Costs  and  Benefits? 

There  was  a  consensus  that  Business  Case  Analysis  (BCA)  language  should  be  used  to 
describe  value  because  the  term  '‘cost"  is  often  associated  only  w  ith  direct  dollar  savings. 
Using  BCA  language  better  implies  the  inclusion  of  measures  that  are  difficult  to 
characterize  in  terms  of  dollars  such  as  increased  reliability  or  reduced  unscheduled 
maintenance.  It  was  indicated  that  the  aircraft  structural  integrity  program  (ASIP) 
descriptions  of  CBM  would  provide  good  insight  and  direction  for  specifically  defining 
costs  and  benefits  as  part  of  a  BCA.  Other  industries  quantify  measures  that  are  not 
directly  related  to  dollars  and  the  SUM  community  may  be  able  to  leverage  this 
knowledge  to  quantify  various  non-cost  performance  measures  for  DoD  or  any  SHM 
applications.  It  was  agreed  that  it  is  important  to  talk  to  the  customer  in  a  language  that 
they  use.  For  instance.  Cost  Effectiveness  Analysis  (CEA)  is  the  term  used  in  the 
propulsion  technology  area.  Using  the  customers'  language  may  help  answer  the  question 
of  how  to  estimate  value  for  benefits  that  are  difficult  to  quantify.  It  was  agreed  that  the 
level  and  application  should  be  identified  to  determine  whether  the  use  of  SUM  systems 
is  beneficial  or  not.  It  was  also  pointed  out  that  the  SHM  community  should  look  at  other 
programs  where  automated  monitoring  technologies  are  being  used,  for  example, 
programs  that  include  rotating  machinery  or  HUMS  systems.  The  structural  monitoring 
community  can  learn  from  those  experiences  about  how  they  performed  a  BCA. 

Are  Cost  Benefit  Studies  Beneficial? 

The  general  consensus  was  that  a  BCA  would  be  required  in  order  to  persuade 
maintainers  and  program  managers  to  advocate  for  new  SHM  technologies.  BCA  should 
be  part  of  an  overall  trade  study  approach.  If  the  magnitudes  are  uncertain,  but  consistent 
across  options,  then  the  relative  differences  can  be  used  to  down-select  specific  options 
for  increased  development  and  analysis.  The  presentation  made  by  Chris  Smith  of  the 
Army  appeared  to  demonstrate  a  BCA  (Smith,  2010),  and  it  would  be  useful  to  take  a 
closer  look  at  what  has  been  implemented  by  the  Army.  Investigating  BCA  cases  about 
how  SHM  technologies  could  be  used  to  manage  an  airplane  past  its  design  limit  is  of 
particular  interest  to  the  Navy  perspective.  The  AFRL  Hot  Spot  program  was  suggested 
as  an  opportunity  where  a  BCA  could  be  demonstrated  in  the  context  of  an  application 
under  study. 

What  Limits  the  Use  of  Cost  Benefit  Studies? 

Many  agreed  that  much  of  the  SHM  community's  advertised/projected  capabilities  have 
not  been  sufficiently  proven.  Thus  it  would  be  very  helpful  to  demonstrate  capability  first 
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using  programs  such  as  the  aging  aircraft.  Acquisition  programs  will  set  requirements, 
but  until  proven  in  the  field,  the  requirements  will  have  limited  insight  for  success.  It  was 
indicated  by  many  participants  that  the  approach  for  program  integration  and  business 
case  demonstration  should  focus  on  taking  incremental  steps  through  small 
demonstrators.  Demonstrations  should  prove  capability,  and  a  BCA  should  be  part  of  the 
process.  The  SHM  community  needs  to  clearly  identify  all  the  possible  benefits 
associated  with  new  SHM  technologies  and  aggressively  evaluate  potential  value 
regardless  of  the  readiness  of  the  technology.  This  would  provide  the  incentive  for 
development  and  application.  Since  specific  applications  provide  limited  value  in  a  larger 
context,  it  was  suggested  that  the  use  of  a  few  representative  business  case  examples 
extrapolated  across  an  entire  weapons  platform  could  provide  a  more  aggregate  business 
case  assessment.  It  should  be  noted  in  making  a  business  case  that  the  complete 
elimination  of  false  calls  is  not  necessary.  What  is  required  is  that  false  negatives  be 
eliminated.  False  positives,  on  the  other  hand,  can  be  tolerated  to  the  point  that  the 
business  case  fails.  Finally,  it  was  pointed  out  that  the  SHM  community  needs  a  good 
success  story  with  demonstrated  benefits.  At  that  point,  other  DoD  programs  will  start  to 
request  their  own  versions  of  the  system  and  ask  for  added  capabilities.  Such  a  success 
will  ease  making  a  business  case  for  other  SHM  systems. 


SUMMARY 

Overall,  it  should  come  as  no  surprise  that  there  is  considerable  interest  in  SHM  within 
the  DoD  as  all  services  spend  a  significant  fraction  of  their  annual  budgets  on 
maintenance  and  support.  However,  it  is  also  true  that  for  every  DoD  platform  both 
already  deployed  and  under  development,  there  is  fierce  competition  for  space  on  the 
vehicle  in  terms  of  weight,  size,  added  capabilities,  and  cost.  Thus,  the  wide  spread 
transition  of  SHM  technology  to  DoD  platforms  will  take  many  years.  However,  it  is 
starting  to  happen  and  there  will  be  opportunities  to  solve  specific  problems  in  the  near 
future.  Finding  these  opportunities  requires  talking  various  DoD  people  including 
weapons  systems  program  management.  R  &  D  program  management,  and  individual 
researchers. 

In  the  meantime,  there  are  particular  steps  in  the  development  process  that  system 
developers  need  to  understand  and  handle  carefully.  These  include: 

1)  Knowing  the  appropriate  language  and  this  includes  a  realistic  assessment  of  TRL 
levels. 

2)  Demonstrated  success  in  testing  on  appropriately  complex  structures  certainly  in  the 
lab  and  probably  under  field  conditions. 

3)  Careful  evaluation  of  capabilities,  costs,  and  benefits  including  some  sense  of  how  the 
benefits  will  be  achieved.  The  consensus  was  that  Business  Case  Analysis  offers  a  better 
approach  than  a  straight  Cost  Benefit  Analysis. 

4)  Understanding  the  needs  and  restrictions  for  each  platform  and  how  the  proposed 
system  would  fit  into  the  maintenance  regime. 

Having  quality  responses  ready  for  these  topics  puts  a  developer  in  position  to  approach 
program  officials  with  a  higher  likelihood  of  success. 
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The  authors  and  workshop  attendees  acknowledge  that  there  is  not  a  single  path  to 
success.  However,  we  feel  that  taking  into  account  the  discussions  and  recommendations 
described  in  this  article  will  help  facilitate  the  transition  of  SHM  from  the  lab  into  full 
acceptance. 
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Table  Captions 


Table  1 :  Proposed  technology  readiness  levels  (TRLs)  for  the  DoD  including  software. 

Table  2:  Development  levels  for  a  number  of  systems  that  sense  mechanical  parameters 
based  on  sampling  rate  criteria. 
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Technology  Readiness 

Level 

Description 

1.  Basic  principles  observed 
and  reported 

IIW/S:  Lowest  level  of  technology  readiness.  Scientific  research  begins  to 
be  translated  into  applied  research  and  development.  Examples  might 
include  paper  studies  of  a  technology's  basic  properties. 

SW:  Lowest  level  of  software  readiness.  Basic  research  begins  to  be 
translated  into  applied  research  and  development.  Examples  might  include  a 
concept  that  can  be  implemented  in  software  or  analytic  studies  of  an 
algorithm's  basic  properties. 

2.  Technology  concept 
and/or  application 
formulated 

HW/S/SW:  Invention  begins.  Once  basic  principles  are  observed,  practical 
applications  can  be  invented.  Applications  are  speculative  and  there  may  be 
no  proof  or  detailed  analysis  to  support  the  assumptions.  Examples  are 
limited  to  analytic  studies. 

3.  Analytical  and 
experimental  critical 
function  and/or 
characteristic  proof  of 
concept 

HW/S:  Active  research  and  development  is  initiated.  This  includes 
analytical  studies  and  laboratory  studies  to  physically  validate  analytical 
predictions  of  separate  elements  of  the  technology.  Examples  include 
components  that  are  not  yet  integrated  or  representative. 

SW:  Active  research  and  development  is  initiated.  This  includes  analytical 
studies  to  produce  code  that  validates  analy  tical  predictions  of  separate 
software  elements  of  the  technology.  Examples  include  software 
components  that  are  not  yet  integrated  or  representative  but  satisfy  an 
operational  need.  Algorithms  run  on  a  surrogate  processor  in  a  laboratory 
environment. 

4.  Component  and/or 
breadboard  validation  in 
laboratory  environment 

IIW/S:  Basic  technological  components  are  integrated  to  establish  that  they 
will  work  together.  This  is  relatively  "low  fidelity"  compared  to  the  eventual 
system.  Examples  include  integration  of  ad  hoc  hardware  in  the  laboratory. 

SW:  Basic  software  components  are  integrated  to  establish  that  they  will 
work  together.  They  are  relatively  primitive  with  regard  to  efficiency  and 
reliability'  compared  to  the  eventual  system.  System  software  architecture 
development  initiated  to  include  interoperability,  reliability,  maintainability, 
extensibility  ,  scalability  ,  and  security  issues,  software  integrated  with 
simulated  current/legacy  elements  as  appropriate. 

5.  Component  and/or 
breadboard  validation  in 
relevant  environment 

IIW/S:  Fidelity  of  breadboard  technology  increases  significantly.  The  basic 
technological  components  are  integrated  with  reasonably  realistic  supporting 
elements  so  it  can  be  tested  in  a  simulated  environment.  Examples  include 
high  fidelity  laboratory  integration  of  components. 

SW:  Reliability  of  software  ensemble  increases  significantly.  The  basic 
software  components  are  integrated  with  reasonably  realistic  supporting 
elements  so  that  it  can  be  tested  in  a  simulated  environment.  Examples 
include  high  fidelity  laboratory'  integration  of  software  components. 

System  software  architecture  established.  Algorithms  run  on  a  processor(s) 
with  characteristics  expected  in  the  operational  environment.  Software 
releases  are  Alpha  versions  and  configuration  control  is  initiated. 

Verification,  Validation,  and  Accreditation  (VV&A)  initiated. 
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6.  System/subsystem  model 
or  prototype  demonstration 
in  a  relevant  environment 

HW/S:  Representative  model  or  prototype  system,  which  is  well  beyond 
that  of  TRL  5,  is  tested  in  a  relevant  environment.  Represents  a  major  step 
up  in  a  technology’s  demonstrated  readiness.  Examples  include  testing  a 
prototype  in  a  high-fidelity  laboratory'  environment  or  in  a  simulated 
operational  environment. 

SW:  Representative  model  or  prototype  system,  which  is  well  beyond  that 
of  TRL  5,  is  tested  in  a  relevant  environment.  Represents  a  major  step  up  in 
software  demonstrated  readiness.  Examples  include  testing  a  prototype  in  a 
live/virtual  experiment  or  in  a  simulated  operational  environment. 

Algorithms  run  on  processor  of  the  operational  environment  are  integrated 
with  actual  external  entities.  Software  releases  are  Beta  versions  and 
configuration  controlled.  Software  support  structure  is  in  development. 

VV&A  is  in  process. 

7.  System  prototype 
demonstration  in  an 
operational  environment 

HW/S:  Prototype  near,  or  at.  planned  operational  system.  Represents  a 
major  step  up  from  TRL  6,  requiring  demonstration  of  an  actual  system 
prototype  in  an  operational  environment  such  as  an  aircraft,  vehicle,  or 
space.  Examples  include  testing  the  prototype  in  a  test  bed  aircraft. 

SW:  Represents  a  major  step  up  from  TRL  6,  requiring  the  demonstration  of 
an  actual  system  prototype  in  an  operational  environment,  such  as  in  a 
command  post  or  air/ground  vehicle.  Algorithms  run  on  processor  of  the 
operational  environment  are  integrated  with  actual  external  entities. 

Software  support  structure  is  in  place.  Software  releases  are  in  distinct 
versions.  Frequency  and  severity  of  software  deficiency  reports  do  not 
significantly  degrade  functionality  or  performance.  VV&A  completed. 

8.  Actual  system  completed 
and  qualified  through  test 
and  demonstration 

HW/S:  Technology  has  been  proven  to  work  in  its  final  form  and  under 
expected  conditions.  In  almost  all  cases,  this  TRL  represents  the  end  of  true 
system  development.  Examples  include  developmental  test  and  evaluation  of 
the  system  in  its  intended  weapon  system  to  determine  if  it  meets  design 
specifications. 

SW':  Software  has  been  demonstrated  to  work  in  its  final  form  and  under 
expected  conditions.  In  most  cases,  this  TRL  represents  the  end  of  system 
development.  Examples  include  test  and  evaluation  of  the  softw  are  in  its 
intended  system  to  determine  if  it  meets  design  specifications.  Software 
releases  are  production  versions  and  configuration  controlled,  in  a  secure 
environment.  Software  deficiencies  are  rapidly  resolved  through  support 
infrastructure. 

9.  Actual  system  proven 
through  successful  mission 
operations 

HW/S:  Actual  application  of  the  technology  in  its  final  form  and  under 
mission  conditions,  such  as  those  encountered  in  operational  test  and 
evaluation.  Examples  include  using  the  system  under  operational  mission 
conditions. 

SW :  Actual  application  of  the  softw  are  in  its  final  form  and  under  mission 
conditions,  such  as  those  encountered  in  operational  test  and  evaluation.  In 
almost  all  cases,  this  is  the  end  of  the  last  bug  fixing  aspects  of  the  system 
development.  Examples  include  using  the  system  under  operational  mission 
conditions.  Software  releases  are  production  versions  and  configuration 
controlled.  Frequency  and  severity  of  software  deficiencies  are  at  a 
minimum. 

Table 
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